Confirmation bias

Confirmation bias is the tendency for decision makers to seek and interpret information in ways that confirm what they already believe. In the account of the Columbia disaster, the bias is not merely an individual error: it becomes an organizational failure, as NASA’s Mission Management Team (MMT) discounts warning signs, declines further observation, and recasts uncertain engineering findings to fit its expectation that the shuttle is safe. The episode contrasts sharply with the book’s argument that groups can make sound judgments when they preserve diverse information and create ways to aggregate it.

How a Prior Conclusion Shapes Inquiry

The MMT approached the question of whether foam could seriously damage the shuttle as though it had already been settled. A technical adviser had been convinced from the outset that foam could not cause serious harm and repeatedly advanced that view. Yet evidence suggested a need for caution. Rather than begin with the evidence and work toward a conclusion, the team reasoned in the opposite direction: its initial assurance shaped what it considered worth investigating.

That assumption affected what information the team sought. The Debris Assessment Team (DAT), whose engineers were evaluating the strike, requested images of the shuttle while it was in orbit. The MMT did not approve the request. Its members believed that the pictures would not have enough resolution to show the impact area, despite lacking the security clearances needed to know what the images could reveal and not asking the Department of Defense about their quality. The book presents this as overconfidence joined to confirmation bias: managers made a consequential decision about evidence they had not established they could assess.

The Columbia Foam-Strike Assessments

During the shuttle’s ascent, a large piece of foam broke off the left bipod area of the external fuel tank and struck the Columbia’s left wing. Launch cameras did not provide a clear view of the impact. The DAT therefore estimated the foam’s size and speed, then used the Crater algorithm to estimate how deeply debris of that size and speed might penetrate the wing’s thermal-protection tiles. By January 21, the engineers had not reached a conclusion, but they had conveyed to Don McCormack that there was reason for concern. In the MMT teleconference that day, the strike was raised late in the meeting, and McCormack described the investigation without transmitting the DAT’s degree of concern. The team leader, Linda Ham, replied that she did not think there was much the team could do, so the strike was not a factor during the flight.

The DAT’s January 24 briefing made the limits of the analysis explicit. The engineers presented five possible scenarios and judged the shuttle likely safe, but stressed that they lacked both good information and adequate tools. Because the MMT had refused the request for on-orbit images, they did not know exactly where the foam had hit. Crater had been designed for debris hundreds of times smaller than the piece that struck Columbia, leaving the reliability of its results uncertain. The MMT nevertheless concentrated on the reassuring conclusion rather than the qualifications that limited it.

McCormack reported that the analysis showed potential for significant damage but no indication of a possible burn-through, while emphasizing uncertainty about the debris’s size, impact location, and angle. Ham then framed the result as meaning no catastrophic damage and no safety-of-flight issue. No MMT member asked to see the DAT study or questioned its uncertainties. By the meeting’s end, the exchange had reduced an unresolved engineering problem to a routine matter of possible tile replacement and turnaround. The book describes the shuttle’s fate as effectively sealed at that point.

Questions That Close Off Alternatives

The book’s clearest illustration of confirmation bias is Ham’s January 22 email asking whether, for any loss of external-tank foam, the team could say that no safety-of-flight damage could occur because of the foam’s density. The desired answer was built into the question: it appeared to invite analysis while steering it toward reassurance. Lambert Austin answered “NO” and warned that the possibility of serious tile damage could not be ruled out. His caution received little attention.

A similar pattern governed the January 24 discussion. Ham repeatedly summarized the analysis in reassuring terms, including by asking whether no burn-through meant no catastrophic damage. When one participant said he had not heard McCormack’s statement, Ham restated it as an absence of a safety-of-flight problem and asked whether there were further questions. The book emphasizes not only what the team concluded, but how its conversational structure narrowed the range of conclusions members could voice. McCormack’s summary omitted the supporting analysis and whether the DAT itself was divided; no one asked for the study or explored the uncertainties. The form of inquiry gave the appearance of discussion while leaving the prior judgment largely undisturbed.

Group Structure and the Wider Argument

The MMT’s confirmation bias operated alongside NASA’s hierarchy. Although NASA was often thought of as meritocratic and bottom-up, the DAT engineers’ concerns—and especially their insistence on seeing the wing before making an informed assessment—did not receive a serious hearing from the managers. The team’s discussions were at once too structured and insufficiently structured: Ham generally asked a question and someone answered, but no process ensured that other members offered their analysis. The book notes that small-group deliberations work better when leaders set a clear agenda and actively make room for every member to speak.

This diagnosis fits the book’s broader distinction between gathering information and making it collectively useful. The intelligence community’s failures before the September 11 attacks are described as failures to recognize the collective significance of information already available. The author cautions that later investigators can identify relevant clues without proving that those clues should have been recognizable beforehand; strategic surprise remains difficult. Still, the repeated inability to anticipate major attacks raises the question of whether different organization and better aggregation might have improved the chance of recognizing a threat. Decentralization can preserve specialized and local knowledge, but without a way to share and aggregate judgments, valuable information may remain isolated. The Columbia case supplies a small-group counterpart: relevant evidence existed, yet the team’s assumptions and procedures kept it from changing the collective assessment.

Contrast with Collective Judgment

Confirmation bias marks one of the conditions under which a group’s judgment can fail; it is not the book’s claim that groups are inherently unreliable. The account of crowd wisdom depends on diversity of opinion, independence, decentralization, and aggregation. When people bring distinct information, form judgments without simply following one another, draw on local knowledge, and have a mechanism for combining their views, individual errors can cancel while useful information remains. The book’s discussion of Scott Page’s simulated problem-solving groups makes a related point: groups combining highly effective agents with less effective but differently skilled agents often outperform groups composed only of the strongest agents. Intelligence alone does not guarantee a range of perspectives.

The contrast is visible in prediction markets. In the wake of the Challenger explosion, investors collectively registered that Thiokol was responsible, even if individual traders were not sure. In NFL betting, the final spread is usually difficult for informed gamblers to beat consistently, and in the Iowa Electronic Markets, election-eve prices across 49 elections from 1988 to 2000 were on average off by 1.37 percent for presidential elections, 3.43 percent for other U.S. elections, and 2.12 percent for foreign elections. These examples do not establish that every crowd is wise; they show how judgments can be comparatively accurate when information is distributed and a process brings it together. At the MMT, by contrast, the conclusion came first, dissent and uncertainty were marginalized, and new evidence was interpreted within the bounds of what the team already believed.